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ABSTRACT 



This paper discusses the validity of test preparation 
practices in the context of large-scale norm- referenced and 

criterion-referenced tests. It presents a framework for conceptualizing the 
various aspects of test preparation that includes the basic premises 
underlying the validity of the practices. Five areas of test preparation are 
outlined: (1) curriculum and test content; (2) assessment approaches and test 

formats; (3) test- taking strategies; (4) timing of test preparation; and (5) 
student motivation. Guidelines and specific test preparation practices 
related to each of the areas are presented with comments about their 
appropriateness and ethnical nature. Political factors influencing the test 
preparation beliefs and practices of educators are described briefly, and 
some conclusions are suggested. A second purpose of this paper is to expand 
the relationship between validity-based judgments about appropriate test 
preparation practices and their implications to the areas of curriculum and 
classroom instruction. If educators can understand test preparation practices 
from an instructional perspective, they can more easily integrate appropriate 
test preparation practices into their regular classroom instruction and focus 
on student learning rather than raising test scores. The difference between 
teaching to the curriculum content objectives and teaching to the test 
depends on the scope of the content domain taught, the focus of instruction, 
and the content focus of the test forms. (Contains 17 references.) (SLD) 
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A Framework for Evaluating the 
Validity of Test Preparation Practices 



Introduction 

The measurement community’s last extended discussions of test preparation practices took place in 
the late 1980's and early 1990's. These discussions occurred shortly after the first round of large-scale 
basic skills achievement testing when these measures began to be used for accountability purposes. 
Educators were faced with pressure to raise test scores. Administrators and teachers increased their 
attention to test preparation and to the burgeoning development of local and commercial test 
preparation practice test materials. 

Several changes have occurred in testing programs and practices since the test preparation discussions 
of almost a decade ago. 

• Most states now have adopted state academic content standards in various subject areas based 
on standards of national professional education organizations. These standards have been 
incorporated into most school district curricula and teachers are required to teach to these 
standards. Many standards include more specific content objectives that represent higher 
levels of learning than were evident in the basic skills curriculum standards of the 1980's and 
early 1990’s. 

• Both norm-referenced tests (NRTs) and criterion-referenced tests (CRTs) have been 
developed on the basis of the content standards of these national education organizations. As 
a result, there is a closer alignment between national, state, and district content standards and 
large-scale assessments than ever before. 

• In some states, CRTs have been “extracted” from NRTs by identifying a subset of items in the 
NRTs that closely match the state content standards and objectives. 

• Some NRTs and CRTs now include a combination of selected-response and constructed- 
response assessment approaches. 

This presentation has two purposes. The first is to examine the validity of test preparation practices 
in the context of large-scale norm-referenced and criterion-referenced tests. A framework for 
conceptualizing the various aspects of test preparation will be presented including basic premises 
underlying the validity of the practices. Five areas of test preparation will be delineated. These 
include (1) curriculum and test content, (2) assessment approaches and item formats, (3) test-taking 
strategies, (4) timing of test preparation, and (5) student motivation. Guidelines and specific test 
preparation practices related to each of the areas will be presented with comments about their 
appropriateness and/or ethical nature that reflect a review of the literature and the basic premises. 
Political factors influencing educators’ test preparation beliefs and practices will be described briefly. 
Finally, conclusions that address the symposium topic will be offered. 



A second purpose of the presentation is to expand the relationship between validity-based judgments 
about appropriate test preparation practices and their implications to curriculum and classroom 
instruction. Often test preparation guidelines are written from the measurement perspective. The 
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guidelines and comments in this presentation are framed to help educators better understand the 
implications of these guidelines from an instructional perspective. In this way, educators could more 
easily integrate appropriate test preparation practices into their regular classroom instruction and 
focus on student learning rather than raising test scores. 

Basic Premises Related to Test Preparation Practices 

This discussion of test preparation practices will examine the validity of such practices in light of the 
changes in tests and test purposes and the changing concepts of validity itself. It is proposed that the 
validity of test preparation practices can be framed with respect to the content and substantive aspects 
of the construct validity of the assessment (Messick, 1994, 1995). The following are basic premises 
used to examine the validity of test preparation practices. ■ 

• Most large-scale assessments are administered to make reasonable inferences about the 
students’ achievement with respect to a domain of content knowledge and/or skills (Popham, 
1991). 

• When test scores are intended to represent achievement with respect to a broad domain of 
content knowledge and skills, test preparation practices should not minimize the accuracy of 
the inference to the broader domain (Mehrens & Kaminski, 1989; Popham, 1991; Popham, 
Mehrens & Ryan, 1998). 

• Test preparation practices should increase students’ mastery of the content domain tested and 
not artificially increase students’ test scores (Mehrens & Kaminski, 1989; Popham, 1991). 

• Test preparation practices should not violate ethical standards of the profession; e.g., theft, 
cheating, lying (NCME Task Force, 1991; Popham, 1991). These practices are often illegal 
and nullify the validity of students’ test results. 

Major Test Preparation Areas 

The various test preparation practices can be categorized into five areas of test preparation compiled 
from a review of the literature and extensive field experiences with district and school administrators 
and teachers. General descriptions of these areas are provided below. 



1. Curriculum and Test Content Test preparation practices that involve the test content 

objectives including the national, state and district 
curriculum standards and objectives on which the test 
was based. 



2. Assessment Approaches and Item Test preparation practices that involve familiarizing 
Formats students with a variety of assessment approaches, (e.g., 

multiple-choice items, short answer items, extended 
response, performance tasks) and item formats within 
each assessment approach (e.g., different types of 
multiple-choice item formats). 
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3. Test-Taking Strategies 



Practices that involve general test-taking or test-wiseness 
strategies unrelated to specific test item content. 



4. Timing of Test Preparation Practices conducted at various points in time before or 

during the administration of the test. 

5. Student Motivation Practices related to motivating students to perform their 

best on the test. 



Each test preparation area will be presented in more detail in the pages thatfollow. In each area, an 
introduction is provided, a general guideline will be given, and specific test preparation practices will 
be listed. Comments about the appropriateness/educational defensibility or ethics/legalities of the 
practice are provided. To clarify the meaning of the comments, definitions of these terms are given 
below focusing on undesirable practices. 



Educationally Indefensible/ 
Inappropriate 



Professionally Unethical/ 
Illegal 



Practices that increase students’ test scores without simulta- 
neously increasing student mastery of the content domain 
tested, i.e., practices that artificially raise students’ scores. 
These practices are not in the student’s best interest because 
inappropriate instructional and other educational decisions 
may be made on the basis of this inaccurate estimate of the 
student’s achievement (Popham, 1991). 

Practices which discredit the profession and/or reduce public 
confidence in public schooling. Some of these practices also 
violate state test security procedures and may have legal 
consequences e.g., loss of credentials (Popham, 1991). 



The comments about the appropriateness and ethics of the various test preparation practices are 
closely related to the changing nature and purposes of norm- and criterion-referenced tests. For this 
reason, several comments may be provided for a particular test preparation practice. Multiple 
comments may be needed because the appropriateness of the practice may depend on different 
circumstances such as the type of test (NRT or CRT) or the size of the content domain assessed in 
the test form. 
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1. Curriculum and Test Content Preparation Practices 



This area of test preparation involves practices related to the curriculum standards and/or content 
objectives on which tests have been developed. Before presenting the general guideline and 
comments about particular test preparation practices, it is useful to examine the meaning of certain 
phrases used in describing content-related practices. Because some measurement specialists and 
educators use the same phrase to refer to different practices, the phrases and their meaning in this 
presentation are provided. 

Teaching to the Curriculum Teaching the objectives or the knowledge and skills in the 

Objectives curriculum content domain defined by the construct. 

Teaching to the Test Teaching/reviewing only the content objectives that are tested 

and including highly-similar test item content. 

Teaching the Test Teaching the actual content/skills in the items of the test. 

Most CRTs and many NRTs assess content that is a part of the national, state, and district curriculum 
standards and objectives. In many states, teachers are required to teach to the standards and 
objectives that represent what students should know and be able to do. Thus, regular classroom 
instruction and assessment which focuses on adopted objectives is test preparation that would include 
teaching to the curriculum objectives. 



In examining test content issues, teaching to the test can have different impacts on instruction and test 
preparation practices depending on how the test specifications are configured. Some of the different 
test configurations are presented below. 



Test 


Number of Forms 


Content Objectives 




Potential Focus 


Configuration 


Administered 


Sampled 


Items in Test 


of Test Prep 


A 


One 


Same 


Same 


Narrow 


B 


Multiple 


Same 


Different 


Narrow 


C 


Multiple 


Different 


Different 


Broad 



Conclusions about the appropriateness of test preparation practices could depend on the number of 
test forms that are administered and released as well as the ways in which the objectives and items in 
the test are sampled. For example, Test Configuration A has the potential impact of narrowing 
instruction and test preparation since one form of a test that contains the same items is administered 
annually. This configuration is typical of many norm-referenced test administrations and presents a 
situation where the most test information is known. Educators can be mosttempted to teach to the 
test or even teach the test since the test specific content objectives are published and the items can 
become familiar to teachers who see the test every year. 

In Test Configuration B, the same objectives are assessed butby different items. This may have the 
potential to prompt teaching to the test by focusing instruction narrowly on the objectives. However, 
there is less potential for teaching the test since the specific content of the items will not be familiar. 



O 

ERIC 



WestEd, Phoenix, AZ - Page 4 



6 



Test Configuration C has the potential to promote a broader focus of instruction and test preparation 
since different specific objectives are being assessed by different items in any one of the multiple test 
forms. This configuration provides the least specific test .information and thus, educators are more 
likely to teach to the curriculum objective domain because they do not know what objectives and 
specific item content will be assessed in any given year. 

It is important to recognize that test developers also play a role that may result in the narrowing or 
broadening of instruction and test preparation. The test specifications and the amount of item 
specification information provided to educators can potentially affect focus of their test preparation 
practices. 

Highly-focused test content instruction and test preparation practices can also have the prospect of 
artificially raising test scores, that is, increasing student scores relative to the specific test content 
without increasing student learning relative to the broader content domain. If such practices are 
effective in raising test scores, they can still be educationally indefensible-detrimental and unfair to 
students who would not have the opportunity to learn other knowledge and skills in a broader content 
domain. Earlier studies involving some commercially-developed materials (Byrd, 1987; Deaton, 
Halpin, and Alford, 1987) have shown no decisive or consistent test score gains of students who used 
such test preparation materials and those who did not. If these materials have not been shown to be 
effective in raising test scores, then they would not pose a threat to the validity of the test results. 
However, since these materials are not effective, then the time spent using them could be better spent 
on regular classroom instruction. 



Guideline 1: Test preparation should be embedded in and focus on teaching the entire 
curriculum objective domain which may include state content standards and appropriate 
norm-referenced test objectives (Mehrens & Kaminski, 1989). 



Test Preparation Practice 

• Teaching, practicing, and assessing the 
entire domain of content objectives in- 
cluding the test objectives, during regu- 
lar classroom instruction. 



Comments 

CRT: Very appropriate and often required if 
these are state standards/objectives which 
will be assessed (Mehrens & Kaminski, 
1989; Popham, 1991). 

NRT: Appropriate if the test objectives are also 
included in the state standards or in the 
district’s curriculum. 

NRT : Inappropriate if the test objectives are not in 
the state standards and/or purposely not 
included in the district curriculum at the 
grade level tested. 
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Test Preparation Practice 


Comments 


• Developing, teaching, and /or practicing 
curriculum objectives based solely on 
the content objectives of the test. 


CRT: Possibly appropriate if the content of the 
test forms is based on state standards which 
represent a broad domain of knowledge and 
skills. 

CRT & NRT: Inappropriate especially when the 
test objectives are relatively limited in scope 
and are intended to represent only a part of 
the content domain or construct (Mehrens 
& Kaminski, 1989; Haladyna, Nolen & 
Haas, 1992). 


• Teaching and/or practicing with actual 
content on previous forms or items of 
the test. 


CRT: Possibly appropriate if the previous forms 
have been released and include a wide range 
of content from the entire content domain 
(Killian, 1992). 

CRT: Inappropriate if the forms have a narrow 
content focus that would not increase mas- 
tery of the content domain tested (Popham, 
1991). 

CRT: Unethical-Illegal if the previous form is not 
released. 

NRT: Inappropriate if the objectives of the pre- 
vious test are not the same for the current 
test and/or if the objectives are not included 
in the district curriculum. 

NRT: Inappropriate if objectives are similar but 
these forms are all that are used to teach or 
review. 


• Teaching and/or practicing with highly 
similar test content on a parallel form 
(locally- or commercially-developed) of 
the current test. 


CRT & NRT: Inappropriate and educationally 
indefensible because such tests could artifi- 
cially raise students’ test scores (Mehrens & 
Kaminski, 1989; Popham, 1991; Haladyna, 
Nolen & Haas, 1991). 


• Teaching and/or practicing actual item 
content on the current form of the test. 


CRT & NRT: Unethical (Mehrens & Kaminski, 
1989; NCME Task Force, 1991; Popham, 
1991; Haladyna, Nolen & Haas, 1992) and 
possibly illegal depending on state test 
security regulations. NOTE: Sometimes un- 
avoidable. There may be a few instances 
particularly in primary grades when the 
content related to an objective is very 
limited, e.g., addition facts with sums to 10. 
In teaching the objective comprehensively, 
teachers will be teaching actual test item 
content (Cohen & Hyman, 1991). 
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2. Assessment Approach and Item Format 
Test Preparation Practices 



Many large-scale assessments now include a combination of selected-response and constructed- 
response assessment approaches such as multiple-choice items, short answer items, and extended 
response tasks. This broadening of the assessment approaches was advanced by educators who 
advocated closer alignments between instruction and assessment. The expanded variety of assessment 
approaches has several benefits: different types of content objectives can be measured in ways that 
yield more valid information about student achievement; and, students have more ways to 
demonstrate different types of learning through different assessment modes. 

Familiarizing students with these assessment approaches and item formats has been generally accepted 
as an appropriate part of test-taking strategy instruction. However, because test preparation 
involving assessment approaches is so closely linked to classroom instruction, this area of test 
preparation is being presented separately from the other test taking practices that will be discussed in 
the next section. 

Using a variety of assessment approaches within classroom instruction has the potential to enrich 
instruction as well as prepare students to take tests that include the various assessment formats. 
Teachers will have more and different types of information about what students know or don’t know. 
All of this information can be very useful for understanding students’ misconceptions and 
determining more effective ways to reteach the content objectives. 

Different approaches and formats can yield different diagnostic information to teachers. For 
example, well-developed multiple-choice items contain alternatives that represent common student 
misconceptions or errors. Short answer item responses can give the teacher information about the 
student’s thinking underlying the answer. 

Classroom assessments and test preparation instruction should also include a variety of item formats 
within each assessment approach. For example, students should have the opportunity during 
instruction to take different types of multiple-choice item formats (including enhanced stimulus 
formats) that assess different levels of cognition (e.g., comprehension, application, analysis, synthesis, 
etc.). 

Generally, performance assessment approaches such as writing assessment are included in tests 
because the curriculum objectives specify the performance. Thus, teaching the objective is test 
preparation involving the performance assessment approach. Students should understand the 
performance criteria (rubrics) and be able to apply them to their own work. The scoring rubrics 
operationalize the curriculum content objectives, i.e., the six- trait writing analytic rubric describes 
various levels of performance of each trait. 
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Guideline 2: Test preparation practices should involve a wide variety of assessment 
approaches, e.g., multiple- choice items, short answer items, extended response, perform- 
ance tasks, especially those that are included in the test (Popham, 1991, Mehrens, 
Popham & Ryan, 1998). Practices should also include a variety of item formats within 
each assessment approach, e.g., different types of multiple -choice item formats. 



Test Preparation Practice 


Comments 


• Administering classroom assessments 
that include a wide variety of assess- 
ment approaches, e.g., multiple-choice 
items, short answer items, extended 
response, performance tasks, espe- 
cially those that are included in the test. 


CRT & NRT: Very appropriate especially if a 
variety of assessment formats are used. 
Students will have the opportunity to 
demonstrate their learning in several ways. 
Teachers will have more information about 
how well students understand and can apply 
their learning in different contexts (Popham, 
1991). 


• Administering classroom assessments 
that include a wide variety of item for- 
mats for each assessment approach. 


CRT & NRT: Very appropriate. Students should 
encounter several different types of 
multiple-choice or performance tasks. For 
example, there are different multiple-choice 
item for-mats to assess place value concepts 
or to add whole numbers in vertical columns 
or horizontal rows or story problems where 
students have to know how to arrange the 
numbers. 


• Identifying and teaching students how 
to apply evaluative criteria (such as 
rubrics) in performance assessments 
(Popham, Mehrens & Ryan, 1998). 


CRT & NRT: Very appropriate. The rubrics are 
part of the task and students should under- 
stand what they are expected to know as 
well as the various “levels" of performance 
to which they can demonstrate their learn- 
ing. 


• Administering practice tests with sam- 
ples of actual item formats used in the 
test. 


CRT & NRT: Appropriate if item format, not the 
content, is the focus of the practice. Many 
large-scale assessments have developed and 
administer short practice tests just before 


/ 


the subject area test are given. 


• Administering classroom assessments 
that only contain item formats from the 
test. 


CRT & NRT: Inappropriate because it places 
undue focus on particular item formats and 
limits the ways in which students can 
demonstrate their learning (Popham, 1991; 
Haladyna, Nolen & Haas, 1992). Refer to 
discussion in the introduction to this section. 
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3. Test-Taking Strategy Preparation Practices 



There is general agreement between measurement specialists and educators that instruction in test- 
taking strategies is important in giving students an equal opportunity to perform — to demonstrate 
their learning. These strategies are important for ensuring that all students are familiar with the basics 
of taking the test and to minimize or eliminate test-taking demands as a source of score variance. 

In their book, A Teacher’s Guide to Standardized Reading Tests, Calkins, Montgomery & Santman 
(1998) describe how their test preparation attitudes and teaching strategies changed as they went 
through a process of rethinking their test preparation practices. 

“...For a great many progressive reading teachers, the standardized test is the enemy, and this 
means that test preparation is too. ..the school or district office would throw a practice test 
booklet at us and we’d throw all this at our kids.. .‘Test practice is not test preparation.' (K. 
Tolan)...our test preparation work was at odds with the rest of our curriculum... we had never 
really considered our most trusted methods of teaching reading to teach children how to read 
tests..." (pp. 69-71) 

The book goes on to describe the reading test-taking strategies they developed and how they 
incorporated these into their regular teaching. It is important to note that many teaching standards 
from national educator associations do not advocate drill and practice strategies in regular classroom 
instruction, particularly when teaching curriculum standards that focus on meaningful learning which 
students can apply. 

Guideline 3: Test preparation should include instruction in and practice of test-taking 
strategies (Mehrens & Kaminski, 1989; Popham, 1991; Haladyna, Nolen & Haas, 1991; Smith, 
1991; Popham, Mehrens, & Ryan, 1998; Calkins, Montgomery & Santman, 1998). 




Test Preparation Practice 


Comments 


• Teaching test- taking skills not related to 
test content such as 

- marking answer sheets; 

- making optimal guesses for certain 
types of items; 

- allocating test-taking time. 


CRT & NRT: Very appropriate. Students should 
be taught test-taking skills to ensure that all 
students are familiar with the basics of taking 
a test. (Mehrens & Kaminski, 1989; Popham, 
1991; Haladyna, Nolen & Haas, 1991; Smith, 
1991; Popham, Mehrens, & Ryan, 1998). 


• Teaching general strategies related to 
subject area test and item formats. 


CRT & NRT: Generally appropriate. Subject 
important in timed tests (Calkins, Mont- 
gomery & Santman, 1998). 





WestEd, Phoenix, A Z - Page 9 

11 



4. Timing of Test Preparation Practices 



Test preparation practices take place at various points in time long before, just before, or during the 
administration of the test. The reco mm endation that test preparation practices involving content 
objectives and assessment approaches and item formats be embedded in regular classroom instruction 
implies that test preparation can and should take place throughout the school year. Cramming 
instruction or test preparation a few weeks before the test administration may do more harm than 
good, especially for lower achieving students. 

Guideline 4: Test preparation should take place throughout the year. 



Test Preparation Practice 


Comments 


During regular instruction (throughout the year) 


• Teaching, reviewing, and assessing the 
curriculum content objectives, includ- 
ing the test objectives, in a variety of 
assessment approaches, and item 
formats. 


CRT & NRT: Very appropriate (Matter, 1986; 
Popham, 1991). High-quality teaching and 
assessment throughout the year are the best 
test preparation strategies. . 


In close proximity to the test (e.g., one month to one week) 


• Teaching and/or reviewing a wide- 
range of content objectives, (which may 
include some test objectives) and a 
variety of assessment approaches and 
item formats. 


CRT & NRT: Generally appropriate. However, if 
too much is taught too close to the test, such 
“cramming” could do more harm than good 
especially for low achieving students.. 


• Administering practice tests that in- 
clude a wide-range of content objec- 
tives = and a variety of assessment 
approaches and item formats. 


CRT & NRT: Generally appropriate. However, if 
students take “practice tests” involving un- 
familiar content, assessment approaches and/ 
or item formats, they may get more confused. 


• Teaching and/or reviewing only the test 
content objectives. 


CRT & NRT: Inappropriate because this practice 
can artificially raise students’ scores 
(Michigan Dept, of Education, 1999). 


• Administering review/practice tests that 
are highly similar parallel forms of the 
test. 


CRT & NRT: Inappropriate because this practice 
can artificially raise students’ scores. 
(Mehrens & Kaminski, 1989; Popham, 1991; 
Haladyna, Nolen & Haas, 1991). 


After seeing the items on the test (just before testing) 


• Teaching or reviewing the actual test 
content a day(s) before students take 
the test. 


CRT & NRT: Unethical and generally illegal (e.g., 
Michigan Dept, of Education, 1999; Ohio 
Dept, of Educ., 1999). 


• Administering practice items that are 
the actual items in the test. 


CRT & NRT: Unethical and generally illegal 
(e.g., Michigan Dept, of Education, 1999; 
Ohio Dept, of Educ., 1999). 
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5. Student Motivation Test Preparation Practices 



Educators and parents are often involved in trying to get their students motivated for school events 
such as athletic or academic Olympiad competitions. Motivating students to take the test seriously is 
at least as critical. 

Many teachers can tell us that when students are not motivated to take tests they often complete a 
test quickly or mark answers randomly without really trying or thinking about their responses to the 
questions. Some students stop responding when they encounter a difficult item. Teachers and 
parents can do much to help students better understand what tests are about and to encourage them 
to do their best and complete the test. 

Guideline 5: Test preparation practices should help students understand the importance 
of doing their best on the test without feeling inappropriately pressured. 



Test Preparation Practice 


Comments 


• Discussing the importance of tests with 
students. 


CRT & NRT: Generally appropriate if the import- 
ance is not over-emphasized so students get 
anxious about taking the test (Nolen, Hala- 
dyna & Haas, 1992). 1 


• Encouraging students to do their best 
and to persevere in completing the test. 


CRT & NRT: Generally appropriate if students 
feel comfortable about what’s expected of 
them. 


• Telling students and/or sending notes 
home to parents about getting a good 
night’s sleep and having breakfast. 


CRT & NRT: Generally appropriate (Smith, 1991; 
Nolen, Haladyna & Haas, 1992). 


• Encouraging attendance during testing 
week. 


CRT & NRT: Generally appropriate (Nolen, Hala- 
dyna & Haas, 1992). If students are sick or 
there is a legitimate emergency, then arrange- 
ments could be made for students to take a 
“make-up” test. 


• Having school pep rallies to promote 
student motivation to do well. 


CRT & NRT: Generally inappropriate. Such 
activities can place undue emphasis on testing 
and decrease instructional time. 


• Telling students that their school can get 
an incentive award if they do well. 


CRT & NRT: Inappropriate. This kind of motiva- 
tion takes the focus off learning and create too 
much pressure on students to do well. 
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Test Preparation Contexts - A View from the Field 



Several studies have shown that some educators and measurement specialists don’t always have a 
common view about the appropriateness or ethical nature of test preparation practices. (Gonzalez, 
1985; Mehrens & Kaminski, 1989; Haladyna, Nolen & Haas, 1991). In trying to understand, not 
necessarily agree with the beliefs and practices of these educators, it is important to consider the 
social and political contexts of testing and the use of test results. Both have an impact on educators’ 
beliefs about the importance of tests, their roles in addressing the mounting pressures of 
accountability, and their test preparation views and practice. These contextual realities and their 
relationship to test preparation practices include the following. 

• High-stakes use of test results can produce extraordinary pressure on administrators and 
teachers to raise students’ test scores. Some examples of high-stakes uses of test results include 
graduation or promotion decisions, money from school incentive awards, or having the school 
and districts scores published in the newspapers 

• Teachers’ and administrators’ knowledge and beliefs about the usefulness or validity of the test 
can influence the kinds and amount of test preparation practices they think are warranted and 
appropriate. 

For example, Smith (1991) concluded from her study that “teachers see fundamental 
discrepancies between true educational attainment and information conveyed by test score 
indicators... Because of these limitations and distortions, the teachers believed that the scores 
were ‘just a number’ or a number so heavily influenced by pupil social class and ethnic group or 
by transitory emotional or volitional states as to be virtually worthless... because tests are so 
distinct from ordinary instruction, teachers believed that test preparation is the only control 
they can exert over test scores.. .Teachers used Scoring High on the ITBS because they believe 
that it is the most effective and efficient method of test preparation, at the least cost of time 
stolen from ordinary instruction” (pp. 538-540). 

Summary and Conclusions 

In summary, the difference between teaching to the curriculum content objectives and teaching to the 
test depends on the scope of the content domain taught, the focus of instruction, and the content 
focus of the test forms. Classroom instruction which focuses on the objectives of the entire 
curriculum domain is not only recommended as appropriate test preparation, it is also required by 
many states that administer standards-based assessments. On the other hand, focusing regular 
classroom instruction and practice only on the test content objectives is considered to be 
inappropriate. It could artificially raise test scores and decrease the validity of the test results and 
resulting inferences and decisions made on the basis of these results. In addition, this narrowing of 
instruction is also unfair to students who may not have the opportunity to learn the larger content 
domain. Teaching the test is regarded as unethical and illegal by most states’ test security policies. 

There are still dilemmas about test preparation that conscientious teachers and administrators raise. 
The higher the stakes of the results, the greater the dilemmas. For example, imagine the feelings and 
thoughts of a high school teacher who has a conscientious student with an otherwise acceptable 
academic record who has failed a high school graduation test by only one or two points and has only 
one more opportunity to take the test. What might be the focus of test preparation that this teacher is 
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considering as he or she reviews the larger issues involved? Hopefully, educators will address these 
dilemmas by making more informed and deliberate judgments about test preparation practices based 
s on the context of the testing situation and the curriculum-, instruction-, and measurement-related 
comments in this presentation. 

This presentation has attempted to provide a educators with a bridge between the validity premises on 
which the test preparation comments are based and the implementation of appropriate preparation 
strategies in classroom instruction. Many teachers believe that test-ing and test preparation take 
valuable time away from regular classroom instruction. They often see test preparation as something 
they do separately from instruction for a period of time before the test administration. Figure 1 below 
portrays this point of view. 



Regular Classroom Instruction 



Test 

Preparation 



Test 

Adm. 



Figure 1. Test preparation separate from regular classroom instruction. 

A major recommendation of this presentation is that high-quality teaching of the content objectives 
and item formats during regular classroom instruction is test preparation of the “best” kind and 
minimizes the need for extended test preparation. Figure 2 below represents this point of view. 



Regular Classroom Instruction 


Test 


Test 


(Test Preparation) 


Prep 


Adm. 



Figure 2. Test Preparation embedded in regular classroom instruction. 

Also emphasized in this presentation is the idea that general test- taking strategy instruction 
throughout the year is more important and can be more effective in enhancing student learning than 
test practice that seeks to raise test scores through strategies such as drill and practice. Embedding 
this test-taking strategy preparation in regular classroom instruction was also recommended. 

In considering the title of this symposium, “Instructionally Corrupt Test-Preparation: Can It Be 
Detected or Deterred, ” we should again to return bridge between validity-based measurement 
guidelines and the implementation of these in schools and classrooms. Too often, it is the 
psychometricians examining “anomalies” in test data who end up detecting inappropriate or unethical 
practices. This is importantif teaching the actual test item contentis involved. However, there are 
many situations in which administrators and teachers may, unknowingly, be teaching to the test 
because they are not fully informed or do not understand the measurement as well as the curriculum 
and instructional aspects of test preparation. Hopefully, the concepts, guidelines, and comments 
presented here can help administrators and teachers better detect both the appropriate and 
inappropriate test preparation practices they advocate and use. Understanding the critical link 
between test preparation and high-quality teaching may help educators refocus their efforts on finding 
ways to better understand the curriculum objectives and expand their repertoire of teaching strategies 
that truly increase student learning. 
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